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(57) Abstract: The present invention relates to an interface for accessing data and, more particularly, to a universal interface for 
accessing data from one or more information systems using a wireless telephone (Fig. 1). The universal interface (200) presents 
the user with an abstraction of a unified mailbox (fax, e-mail, voice mail, among others) and advanced web services from a wireless 
telephone. It does not lock the individual into one information provider, and allows the user to access multiple information providers 
with a single telephone call. It also provides a simple system that acts as a conduit, translating an individual's spoken or tone com- 
mands into commands recognizable by the various information systems (110). 
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UNIVERSAL INTERFACE FOR VOICE ACTIVATED ACCESS 
5 TO MULTIPLE INFORMATION PROVIDERS 

BACKGROUND OF THE INVENTION 
The present invention relates to an interface for 
accessing data and, more particularly, to a universal 

10 interface for accessing data from one or more information 
systems using a wireless telephone. 

As the content available from information providers 
expands, so does the complexity of retrieving data. For 
example, prior to the widespread use of electronic mail (e- 

15 mail) and alternate telecommunication systems, individuals 

would typically leave messages on and retrieve messages from 
telephone answering machines. With the numerous message 
centers available today, messages may be scattered over 
various telephone Messaging systems, electronic mail 

20 accounts, and facsimile machines . Moreover, on a daily, if 
not hourly, basis, individuals are storing information on, 
and retrieving information from, personal information 
management systems (PICS) , electronic office managers,, and 
world wide web. sites. 

25 The availability of these information systems renders 

information, formerly hard, to manage and retrieve, readily 
accessible. Aiding the accessibility of this data is the 
fact that much of the data, some of which was previously 
available only in hardcopy, is now stored electronically and 

30 is thus searchable and retrievable. Therefore, individuals 
are able to access a multitude of data from multiple 
information providers and from the convenience of their 
. computers . 

To date, the advantages associated with the widespread 
35 availability of electronic data have not been fully 

realized. At least three drawbacks exist with respect to 
the manner in which information currently is stored. First, 
not all information is stored or available in the same 
format. For example, some information is stored as speech, 
40 others are stored as text. Second, some information is 

available only in text form, and would thus be inaccessible 
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to a traveler or a mobile user who needs to access 
information using his or her wireless phone. Third, even 
the if information is available in the- same format- and 
accessible to wireless telephones, the information is not 
centralized, but is generally offered by a number of 
information providers. Thus, if one sought access to 
various data, he or she would need to access numerous 
information systems. 

SUMMARY OF THE INVENTION 

A universal interface accesses one or more information 
systems from a telephone, each of the information systems 
being adapted to generate text data or speech data in 
response to a query. The universal interface includes an 
interface control module for receiving a sound-based command 
from a user and for sending a corresponding machine readable 
command to a source selected from one of the information 
systems, the interface control module receiving data from 
the source in response to the corresponding machine readable 
command; a text-to-speech converter coupled to the interface 
module for converting text data received by the interface 
control module to a converted speech segment and playing the 
converted speech segment for the user; and a router coupled 
to the interface module for receiving speech data from the 
source and playing a speech segment corresponding to the 
speech data for the user. 

Implementations of the system may include one or more 
of the following. The telephone can be wireless. The 
sound'based command can come from the user's voice, or can 
be specified using one or more telephone touch-tones. 

In another aspect, a universal interface accesses one 
or more information systems from a user telephone. The 
universal interface includes an input converter for 
converting input from the user telephone to commands; an 
interface control module coupled to the input converter for 
receiving the commands from the input converter; determining 
one of the information systems to be accessed; converting 
the commands to commands recognizable by the information 
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system; forwarding the converted commands to the information 
system; receiving data from the information system; 
detecting the form of the data from the- information system; 
and storing information relating to a current state of the 
system. The universal interface also includes a 
speech'to-text routing switch coupled to the interface 
control module for receiving data from the information 
system and control data from the interface control module; a 
speech- to- command converter coupled to the interface control 
module for converting speech to commands, wherein the 
speech-to-command converter coupled to the speech-to-command 
routing switch to receive speech and to forward commands to 
the interface control module; an output switch coupled to 
the interface control module and the speech- to- text routing 
switch for receiving speech from the speech- to- text routing 
switch for receiving a control input from the interface 
control module, and for forwarding speech from the 
speech- to- text routing switch to the user • telephone ; and a - 
text -to- speech converter coupled to the output switch for 
receiving text from the interface control module, converting 
the text to speech, and forwarding the speech to the output 
switch to deliver speech to the user telephone. The user 
telephone is a mobile telephone. The input converter can be 
a speech-to-command converter, a tone -to -command converter. 
The interface control module integrates the data from the 
information system periodically or manually under user 
control and includes: means for retrieving data from the 
information systems; means for determining antecedent 
comparable relevance of the data; means for updating all of 
the data to reflect the most recent data; means for linking 
relevant data; and means for exporting the linked data to 
the information system. 

Implementation of the above aspect may include one or 
more of the following. The information systems can include 
two or more of the following: a calendar; a to-do list; an 
address book; voice mail; e-mail; and a web site. The 
interface control module integrates and synchronizes (i)a 
database of a personal information manager, (ii) a database 
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residing on a personal digital assistant, and (iii) a 
database residing in the universal interface, the universal 
interface coupled to' a* computer on which the personal ' 
information database resides. A computer further comprises: 
a first input terminal for receiving data from the personal 
digital assistant; a second input terminal for receiving 
data from the universal interface; a sensor for detecting a 
synchronization event triggered by a user requesting 
synchronization of the database of the personal digital 
assistant with the database of the personal information 
manager; an electronic mail system coupled to the Internet; 
and control logic coupled to receive the synchronized event 
from the sensor and to transmit data, over the electronic 
mail system, to the interface control module, wherein the 
control logic updates the data in each of the databases to 
reflect the most recent data entered into any database. 
The synchronization information can be sent to the universal 
interface in its entirety, in compressed form, or in 
incremental form. The. data sent from the information system 
to the universal interface can be sent over the Internet, 
The data sent can be encrypted. The output switch can be a 
detector for determining whether the speech from the 
speech-to-text routing switch is digital or analog; a sound 
card coupled to the detector to receive digital data; a 
first speech routing switch for receiving a control input 
from the detector, and speech from the detector or speech 
from the sound card; a second speech routing switch for 
receiving the control input from the interface control 
module and speech from the first speech routing switch or 
speech from the text-to-speech converter, the second speech 
routing switch forwarding speech to the user telephone. The 
information system can include at least one of: a dual tone 
multiple frequency (DTMF) driven voice mail system, a voice 
driven voice mail system, an electronic mail system, a web 
site, and a personal information manager. The system can 
determine one of the following: whether the voice commands 
are being received from a user telephone; the information 
system to be accessed; whether the voice commands, after 
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being converted to commands, are recognizable to the 
information system; whether the converted commands have been 
forwarded to the information system; whether data has been 
received from the information system; whether data from the 
5 information system is speech or text; the state of the 

speech-to-text routing switch; and the state of the output 
switch. 

The interface control module further includes a model 
for converting commands from the telephone into commands 

10 recognizable by the information system; and a translator 
coupled to the input converter for retrieving the model 
corresponding to the information system to be accessed and 
for converting the commands to commands recognizable by the 
information system. The universal interface can include 

15 means coupled to the input converter for signaling that the . 
user telephone has received unintelligible words; means for 
restarting communication to the text-to-speech converter at 
a point a specified number of words back from the point at ■ 
which the communication ceased; means for forwarding the 

20 first specified number of words by spelling the v/ords out; 
and means for continuing forwarding the data after the 
specific number of words. The universal interface can 
include means for detecting a first language in which the 
commands from the user telephone are received; means for 

25 detecting a second language associated with the data 
received from the information system; and means for 
converting the data from the information system into the 
first language. The system can detect more than one 
languages within a single fragment of data. A resource 

30 manager for establishing conference bridges to an external 
telephone having a telephone number, wherein the interface 
control module detects, from the commands from the input 
converter, whether a conference bridge request has been 
made; the interface control module retrieves the telephone 

35 number of the external telephone to establish the conference 
bridge; the interface control module forwards the telephone 
number to the resource manager; and the resource manager 
establishes a telephone connection with the external 
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telephone, A facsimile manager can be used for sending 
facsimiles to one or more facsimile machines, wherein: the 
interface control module detects, from the commands from the 
input converter, whether a facsimile request has been made; 
5 the interface control module retrieves a telephone number of 
a designated facsimile machine; the interface control module 
forwards the data and the telephone number to the facsimile 
manager; and the facsimile manager faxes the data to the 
designated facsimile machine , 

10 . The universal interface can have a pager manager for 

sending pager messages to one or more pagers, wherein the 
interface control module detects, from the commands from the 
input converter, whether a pager request has been made; the 
interface control module retrieves data to be forwarded to 

15 the pager; the interface control module retrieves a 

telephone number of a designated pager to which the data is 
to be forwarded; the interface control module forwards the 
data and the telephone number to the pager manager; and the- 
pager manager forwards the data to the designated pager. 

20 In another aspect, a universal interface for accessing 

data from one or more information systems includes an input 
converter for receiving inputs from the user telephone and 
converting the inputs into commands; and an interface 
control module coupled to the input converter. The 

25 interface control module includes means for periodically 
requesting data from the information systems; means for 
retrieving data from the infoirmation system; means for 
detecting the form of data from the information system; 
means for converting the data that is speech to commands; 

30 means for storing the data for later access by a user 

telephone; and means for storing information representing a 
current state of the system; and further including means for 
receiving commands from the input converter and means for 
detecting which of said stored information is sought to be 

35 . accessed; 

a text-to-speech converter coupled to the interface 
control module for receiving said requested stored text from 
the interface control module, converting the text to speech. 
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and forwarding the speech to the user telephone. 

-The input converter can he a - speech- to -command- converter or • 
a tone -to -command converter. The interface control module 
5 integrates the data from the information system periodically 
or manually under user control, and can include means for 
retrieving data from the information systems; means for 
determining antecedent comparable relevance of the data; 
means for updating all of the data to reflect the most 

10 recent data; means for linking relevant data; and means for 
exporting the linked data to the information system. 

The universal interface of claim 25, wherein the 
interface control module integrates and synchronizes (i) a 
database of a personal information manager, (ii) a database 

15 of a personal digital assistant, and (iii) a database 

residing in the universal interface, the universal interface 
coupled to a computer on which the personal information 
database resides. The computer can include a first input 
terminal for receiving data from the personal digital 

20 assistant; a second input terminal for receiving data from 
the universal interface; a sensor for detecting a 
synchronization event, wherein the event is triggered by a 
user requesting synchronization of the database of the 
personal digital assistant with the database of a personal 

25 information manager; an electronic mail system coupled to 
the Internet; control logic coupled to receive detection 
information from the sensor and to transmit data, via the 
electronic mail system, to the interface control module, 
wherein the control logic updates the data in each of the 

30 databases to reflect the most recent data entered into any 
one database. The synchronization information is sent to 
the universal interface in its entirety, in compressed form, 
or in incremental form. The data sent from the information 
system to the universal interface is sent over the Internet. 

35 The data sent over the Internet is encrypted. The 

information system can be a dual tone multiple frequency 
(DTMF) driven voice mail system, a voice driven voice mail 
system, an electronic mail system, a web site, and a 
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personal information manager. The system can determine 
whether the voice commands are being received from a user 
telephone; the information system to be accessed; whether 
the voice commands, after being converted to commands, are 
recognizable to the information system; whether the 
converted commands have been forwarded to the information 
system; whether data has been received from the information 
system; whether data from the information system is speech 
or text; the state of the speech-to-text routing switch; and 
the state of the output switch. The interface control 
module includes one or more models containing commands 
recognizable by the information system; control logic for 
accessing the model that corresponds with the information 
system to be accessed, converting control commands to 
commands recognizable by the information system, and 
forwarding the converted commands to the information system. 
The universal interface includes means coupled to the input 
converter for signaling that the user telephone has received 
unintelligible words; means for restarting communication to 
the text-to-speech converter at a point a specified number 
of words back from the point at which the communication 
ceas;ed; means for forwarding the first specified number of 
words by spelling the words but; and means for continuing 
forwarding the data after the specific number of words. The 
universal interface can include means for detecting a first 
language in which the commands from the user telephone , are 
received; means for detecting a second language associated 
with the data received from the information system; and 
means for converting the data from the information system 
into the first language. The universal interface further 
includes means for detecting more than one language within a 
single fragment of data. 

In another aspect, a method provides data from one or 
more information systems to a user telephone by converting 
speech from the user telephone to commands; determining the 
information system to be accessed; converting the text to 
commands recognizable by the information system; forwarding 
the converted commands to the infoinfnation system; receiving 
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data from the information system; detecting the form of the 
data from the information system; converting non- speech data 
from the information system into speech; forwarding the 
speech data to the user telephone; and storing information 
relating to the current state of the system. 

Implementations may include one or more of the 
following. The converting speech from the user telephone can 
include converting speech from a mobile telephone, 
converting speech to commands, or converting 
dual-tone-multiple-frequency (DTMF) information to commands. 
The data can be synchronized from the information systems by 
periodically, or upon command: retrieving data from the 
information systems; comparing the data to determine which 
data is most recent; updating all of the data to reflect the 
most recent data; linking relevant data; and exporting the 
linked data to the information systems. 

The information systems can include two or more of the 
following: a calendar; a to-do list; an address book; voice 
mail; e-mail; and a web site. The method includes 
integrating and synchronizing (i) a database of a personal 
information manager, (ii) a database of a personal digital 
assistant, and (iii) a database residing in a universal 
interface, by: receiving data from the personal digital 
assistant; receiving data from the universal interface; 
detecting a synchronization event, wherein the event is 
triggered by a request for synchronization of the database 
of the personal digital assistant with the database of a 
personal information manager; transmitting data, via an 
electronic mail system, to the interface control module; and 
updating the data in each of the databases to reflect the 
most recent data entered into any one database. The 
synchronization information can be sent to the universal 
interface either in its entirety, in compressed form, or in 
incremental form. The data can be sent from the information 
system to the universal interface over the Internet. The 
method includes encrypting the data from the information 
system before sending the encrypted data over the Internet. 
The infoormation system can include at least one of: a voice 
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mail system, and an electronic mail system, a web site, and 
a personal information manager. The method includes storing 
information relating to the current state of the system 
further including determining whether the voice commands are 
5 being received from a user telephone; the information system 
to be accessed; whether the voice commands, after being 
converted to commands, have been converted into commands 
recognizable by the information system; whether the 
converted commands have been forwarded to the information 

10 system; whether data has been received from the information 
system; whether data from the information system is speech 
or text; the state of the speech- to- text routing switch; and 
the state of the output switch. 

The converting of text to commands recognizable by the 

15 information system can include accessing one or more models 
for converting commands from the telephone into commands 
recognizable by the information system. The method also 
includes receiving a command from the user telephone 
signaling that the user telephone has received 

20 unintelligible words; ceasing communication to the user 

telephone; restarting communication to the user telephone at 
a point a specified number of words back from the point at ' 
which the communication ceased; forwarding the first 
specified number of words by spelling the words out; and 

25 continuing to forward the remainder of the data. The method 
can also include detecting the language in which the 
commands from the user telephone are received; detecting the 
language of the data received from, the information system; 
converting the data from the information system into the 

30 language in which commands from the user telephone are 
received. 

The detecting the language of the data received from 
the information system further includes detecting more than 
one language within a single piece of data. The method 
35 includes establishing conference bridges to one or more 

external telephones, by: detecting, from the commands from 
the input converter, whether a conference bridge request has 
been made; retrieving a telephone number of the external 
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telephone to which the conference bridge is to be 
established; forwarding the telephone number to a resource 
manager; and establishing, through the resource manager, a 
telephone connection with the external telephone, thereby 
5 opening a line of communication between the user telephone 
and the external telephone. The method includes sending 
facsimiles to one or more facsimile machines, by: detecting, 
from the commands from the input converter, whether a 
facsimile request has been made; retrieving data to be 

10 faxed; retrieving a telephone number of a designated 

facsimile machine to which the data is to be forwarded; 
foirwarding the data and the telephone number to a facsimile 
manager; and with the aid of the facsimile manager, faxing 
the data to the designated facsimile machine. The method 

15 also includes sending pager messages to one or more pagers, 
by: detecting, from the commands from the input converter, 
whether a pager request has been made; retrieving data to be 
forwarded to a pager; retrieving a telephone number of a 
designated pager to which the data is to be forwarded; 
. 20 forwarding the data and the telephone number to a pager 

manager; and with the aid of the pager manager, forwarding 
the data to the designated pager. 

In another aspect, a method accesses data from one or 
more information systems by retrieving data from the 

25 information system; detecting the form of data from the 

information system; converting the data that is speech to 
text; storing the data for later retrieval; and stqring 
information representing the current state of the system. 

Advantages of the invention include one or more of the 

30 following. The invention provides a seamless access to one 
or more information systems. More particularly, the 
invention provides access to multiple communication systems, 
some of which are text-driven, using a wireless telephone. 
The Universal Interface ("UI") of the invention recognizes 

35 voice commands, converts the commands into commands 

recognizable by the information systems, and utilizes the 
converted commands to access information systems and 
retrieve data therefrom. The UI enables an individual to 
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retrieve all of his or her information using a single device 
such as a wireless telephone. For instance, the individual 
can retrieve various information from a wireless telephone 
including voicemail messages, personal information manager 
5 information, to do lists, electronic mail messages, and 
information stored on the Internet. 

The UI presents the user with an abstraction of a 
unified mailbox (fax, e-mail, voice mail, among others) and 
advanced web services from a wireless telephone. It does 

10 not lock the individual into one information provider, and 
allows the user to access multiple information providers 
with a single telephone call. It also provides a simple 
system that acts as a conduit, translating an individual's 
spoken or tone commands into commands recognizable by the 

15 various information systems. 

BRIEF DESCRIPTION OF THE DRAWINGS 
Fig. 1 is a block diagram illustrating a universal 
. interface for a mobile user. 
20 Fig. 2 is a block diagram showing the universal 

interface for accessing the Internet. 

Fig. 3 is a block diagram illustrating a first 
embodiment of the universal interface for processing speech 
commands . 

25 Fig. 4 is a block diagram illustrating a second 

embodiment of the universal interface for processing touch- 
tone input . 

Fig. 5 is a block diagram illustrating a third 
embodiment of the universal interface, with resource, 
30 facsimile, and pager managers. 

Fig. 6 is a block diagram showing the universal 
interface pf Fig. 3 for communicating with a personal 
information manager C'PIM"). 

Fig. 7 is a block diagram illustrating an interface 
35 control module. 

Fig. 8 is a block diagram illustrating an output switch 
of the universal interface . 

Fig. 9 is a block diagram illustrating a speech-to-text 
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routing switch of the universal interface. 

Fig. 10 is a block diagram illustrating various ways of 
accessing the information systems using the universal 
interface . 

5 Fig. 11 is a block diagram illustrating automatic data 

retrieval and non-real-time access to the universal 
interface using voice commands. 

Fig. 12 is a block diagram illustrating automatic data 
retrieval and non-real-time access to the universal 
10 interface using touch-tone inputs. Fig. 12 includes 
language converter and spell converter options. 



DESCRIPTION 

15 Referring now to Fig. 1, a universal interface 200 for 

mobile users is shown. A user communicates with an 
information system 110 using a telephone 105 which may be 
wireless. A universal interface 200 receives user commands 
from the wireless telephone 105 and converts those commands 

20 into commands recognizable by the information system 110. 

The universal interface 200 then sends those commands to the 
information system 110. The information system 110 responds 
to the user commands and sends the responsive information to 
the universal interface 200. The universal interface 200 

25 converts the data from the information system 110 if the 
data is not in a format recognizable by the wireless 
telephone 105. Finally, the universal interface 200 
provides the data to the mobile telephone 105, thus 
providing the data to the user. This process of retrieving 

30 information from the information system 110 through the 

universal interface 200 is repeated until the user retrieves 
all the required data. 

The universal interface 200 can be used to access 
various data in addition to voice mails and e-mail. For 

35 example, as shown in Fig. 2, the user can access Internet 

data such as stock quotes and weather reports from web sites 
125 and 130 on the Internet 120. These data are collected 
and processed by a web server using intelligent web server 
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agent tools such as the webMethods B2B, extensible Markup 
Language (XML) - based e-commerce solution, available from 
webMethods, Inc. of Fairfax, Virginia. 

WebMethods B2B automates data exchange between 
5 different applications, Web sites and legacy data sources. 
WebMethods B2B employs Web Interface Definition Language 
(WIDL) , which enables the characterization of automated 
accesses to Web-enabled resources and other applications 
through well-defined interfaces. By abstracting such 

10 information, the applications can continue to be accessed 
regardless of changes in the location, appearance, and 
structure of the underlying data. 

Alternatively, data may be collected and processed by a 
web server using VoxML Voice Mark-up Language, available 

15 from Motorola, Inc. VoxML is a mark-up language designed 

specifically for voice applications on the Web. It is based 
on the W3C XML standard and follows the syntactic rules of 
XML. Further, VoxML offers the advantage of allowing for 
the querying of web servers through the use of voice 

20 commands sent over a telephone. The VoxML language can be 
used to express multi-step dialogues that collect commands, 
input values, and audio samples from a user, and play 
recorded audio and synthesized speech back to the user. 

In one embodiment, to communicate with the user, the UI 

25 200 is connected to a cellular switch over one or more Tl 
lines. This connection allows a mobile telephone user to 
dial into the cellular switch, and concomitantly into the UI 
200. 

The UI 200 uses the D240 Tl platform, available from 
30 Dialogic Corp. of Parsippany, NJ. Connectivity to the 
cellular switch can be via SSI or E&M MF/DTMF Channel 
Associated Switching, where the UI 200 can talk to the 
switch using SS7 point codes. Ports can be universal, so 
that they can function either as incoming or outgoing ports 
35 using dynamic reallocations. The universal interface 200 

may operate on a scalable and fault -tolerant computer system 
such as that disclosed in copending U.S. Application Serial 
No. 09/169,838, entitled ^^Server Pool for Clustered System," 
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filed October 9, 1998 and commonly assigned, hereby 
incorporated by reference; and as that disclosed in 
copending U.S. Application Serial. No. 09/169,361, entitled 
Fault Tolerant Bus for Clustered System," filed October 9, 
5 1998, and commonly assigned, hereby incorporated by 
reference . 

Fig. 3 illustrates the internal structure of the 
universal interface. If the input to the universal 
interface 200 is voice, the voice is first converted to 

10 computer-recognizable commands via a speech-to-command 
converter 220. The converter 220 uses the Speech 
Recognition Technology software, available from Nuance 
Communications of Menlo Park, California. The 
implementation can either be host-based or line card-based. 

15 Once the speech has been converted to computer- 

recognizable commands, the converted commands are then 
provided to an interface control module {^'ICM") 300. The 
interface control module 300 receives commands from the 
speech-to-command converter 220 and determines which 

20 information system the user is attempting to access. It 

then converts the commands into commands recognizable by the 
information system 110. The interface control module 300 
then establishes contact with the information system 110. 
Once contact has been established, the interface control 

25 module sends converted commands to the information system. 
These converted commands are processed by the information 
system in the same manner as if the user was accessing the 
information system directly. The information system 110 
receives the commands and provides data in response. The 

30 data generated by the information system 110 is provided to 
both the interface control module 300 and a speech-to-text 
routing switch 500. The interface control module 300 
determines whether the data from the information system 110 
is speech or text. The interface control module 300 then 

35 sends a control signal to the speech-to-text routing switch 
500. This control signal indicates whether the speech-to- 
text routing switch 500 should be toggled. 

In a toggled state, the data from the information 
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system lio is sent directly through the speech-to-text 
routing switch 500 into an output switch 400. The speech- 
to-text routing switch 500 is only toggled, and thus only 
sends data directly to output switch 400, when the interface 
5 control module 3 00 detects that the data from the 

information system 110 is speech. The data sent directly to 
output switch 400 is also sent to speech-to-command 
converter 260. The speech- to- command converter 260 converts 
the speech into commands and sends the commands to the 

10 interface control module 300. The ICM 300 uses these 
commands to determine the state of the system. 

If, on the other hand, the interface control module 300 
detects that the data from the information system 110 is 
text, the interface control module will route the text to 

15 the text-to-speech converter 230, and will send a control 
signal to output switch 400, routing the speech, from the 
text-to-speech converter 230 to the output of the universal 
interface 200. 

The text-to-speech converter 230 automatically produces 
20 speech through transcription. The text-to-speech converter 
contains two modules- -one that processes natural language 
and a second that performs digital signal processing. The 
first module converts text into its phonetic equivalent. 
The second converts the output of the first into speech. 
25 In the alternative, a conventional voice response system can 
be employed. 

Also, in the embodiment the text -to- speech engine may 
operate on a D240 Tl line card (which is available from 
Dialogic). Also, Accuvoice's or Lernout & Hauspie's 

30 TruSpeech software can be used to handle speech. 

Also, in one embodiment, a VUI (Voice User Interface) 
indicates various phrases that the system will recognize, 
and a VAV (Virtual Assistant Voice) indicates prompts and 
information from voice messages, PIM or the Web that the 

35 system will play. The VUI allows the user to speak 

naturally to request the information the user needs at any 
level. However, when the user wants to enter information to 
the speaker, the user initiates a step-by-step process to 
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interact with the system. 

For example, a user's first command may take the form 

of: . 

^'I would like to access my [menu item]/* where menu item 
5 might be calendar, to do list, address book, auto wakeup, 
web information, or another information system, where words 
not between brackets are optional. 

Once the user has accessed one of the menu items, 
typical user commands might include: 
10 Play mail : " [Play, Brief ] [priority] [E-mail, voice 

mail] from [person]," where priority can be urgent or 
normal, among others. 

Address book look-up : "I would like to [Find, lookup] 
[name]," where name is the person's name the user wants to 
15 find. 

• Call management : "[I would like to] [call, dial, make a 
conference call to] [Name, phone number]," where name is the 
person's name the user wants to call. 

Appointment and to do list lookup : would like to 
20 [find, check] My [appointment, to do] for [date]," where 
date can be tomorrow or any day of the week. 

When the user utters these commands to the system, the 
system looks for information to complete the commands. If 
the system cannot complete the commands, the UI 200 prompts 
25 the user step-by-step for the required information. For 

example if the system registers the appointment keyword, but 
not the date, the UI 200 prompts the user for only the date. 
The UI 200 might state, ''Which appointment date do you want 
me to check?" 

30 Just as the UI 200 allows the user to access 

information, it operates similarly to allow the user to 
communicate information. In one embodiment, the UI 200 has a 
second level menu structure. This second level structure, 
in response to the user's commands, prompts the user for 

35 information. For example, the initial user command might 
be: ''I would like to [maJce, delete, update] [an entry, an 
item, a name] If the user chooses to update an address 
entry, the UI 200 may prompt the user as follows: 
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'*What is the business phone number of the person?" 
[user speaks business phone number] 
**What is the home phone number of the person?" 
[user speaks home phone number] 
5 In one embodiment, the UI 200 can repeat the user 

information to ensure that the information has been entered 
correctly. In another, the user can query the number of 
entries in the address book. 

The user can also enter entries to an appointment 
10 calendar. In this instance, the UI 200 may prompt the user 
for a date of the appointment, a beginning time of the 
appointment, a duration of the appointment, and an agenda. 

The user can also provide entries to the to-do list. 
The UI 200 might then provide prompts for the due date, 
15 • priority, and agenda. If the user prompts the UI 200 to 

enter an autowakeup, the UI 2 00 may prompt the user for the 
date of the entry and the time of the entry, and the phone 
number to dial at the wake -up time. 

If the user prompts the UI 2 00 to set up a conference 
20 call, the UI 200 may prompt the user as follows: ^*Who do you 
want to conference in? Say the name of the persons or phone 
number. When you are finished say 'done.'" After each time 
the user speaks a name or number, the UI 200 responds with 
''Got it. Who is next?" The UI 200 continues to prompt the 
25 user until the user indicates that he or she is done 
entering names or numbers. 

In another embodiment, if an incoming call occurs 
during the conference call, the UI 2 00 indicates to the user 
that there is an incoming call. For example, the UI 200 may 
30 say "[Name] is on the other line, do you want to take it?" 
The user may then answer: "[I will take it, conference her 
in, voice mail, (ignore virtual assistant)]." 

Just as the user can update information, he or she can 
also delete or modify entries. If the user attempts to 
35 delete information, the UI 200 may prompt the user to verify 
the request prior to deleting the entry. And, just as in 
conventional systems, certain use preferences such as 
whether or not to inquire before deleting may be stored in 
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memory . 

Further, once the user has prompted the UI 200 to 
perform a particular function, the UI 200 can- prompt the 
user to ensure that the user does not want to repeat a 
5 similar function. For example, once the user has prompted 
the UI 200 to enter an item into the to-do list, the UI 200 
might prompt the user to determine if the user wants to add 
another entry. 

In one embodiment, the UI 200 has additional functions 

10 related to the playing or briefing of messages. These 

commands include: save, delete, play, forward, reply, next, 
previous, higher volume, lower volume, skip forward, skip 
backward, faster, slower. 

In Fig. 3, the text -to -speech converter 230 routes 

15 speech to the output switch 400. The interface control 

module 300 sends a control signal to the output switch 400. 
In a non- toggled state, the output switch 400 receives the 
output of the text-to-speech converter 230. In a toggled 
state, the output of the output switch 400 receives the 

20 output of the speech-to-text routing switch 500. 

If the data from the information system 110 is speech, 
it is routed directly to the output of the universal 
interface 200. If, on the other hand, the data from the 
information system 110 is text, the data is converted to 

25 speech via the text -to -speech converter 230 and is then 

routed through the output switch 400 to the output of the 
universal interface 200. 

Fig. 4 is similar to Fig. 3, except that the input to 
the universal interface 200 handles touch- tone inputs rather 

30 than voice inputs. Therefore, the speech- to -command 

converter 220 of Fig. 4 is replaced with a tone -to -command 
converter 250 of Fig. 4. The tone-to-command converter 250 
converts touch-tone inputs from a telephone of the user into 
commands. The commands are presented to the interface 

35 control module 300. The interface control module 300 then 
functions as described above. 

Fig. 5 illustrates a universal interface with resource, 
facsimile, and pager managers. These managers are 
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accessible from the web. 

The universal interface 200 accepts touch-tone from the 
user's telephone. This tone is converted subsequently to 
commands through the tone -to -command converter 250. As in 
Fig. A, the output of the tone- to -command converter 250 is 
then sent to the interface control module 300. The 
universal interface 200 functions in the same manner as the 
universal interface 200 of Fig. 4, except that the universal 
interface 200 of Fig. 5 can support resources, facsimiles 
and pagers through a resource manager 270, a facsimile 
manager 280, and a pager manager 290. These managers 
interface with an external telephone 275, a facsimile 
machine 285, and a pager 295, respectively. 

The pager manager 290 can receive pages from another 
user, and can also page one or more external pagers 295. 
The pager manager 2 90 sends the information received from 
the pager 295 to the interface control module 300. The 
pager manager manages pager and message delivery queues. 
The ai 200 may use a native NT file system or CIFS virtual 
file system to manage system wide pager and dial-out queues. 
See, for example, copending U.S. Application Serial No. 
09/169,360, entitled ^'Shared- Everything File Storage for 
Cluster System," filed October 9, 1998, and commonly 
assigned, hereby incorporated by reference. 

Similarly, the facs imile manager 280 receives facsimile 
messages from another user, and sends faxes to the facsimile 
285. The fax manager manages the queues for facsimiles. It 
keeps track of the number of facsimile resources available 
in the system, maintains a list of outgoing faxes to be 
delivered, and incorporates a retry mechanism in case of an 
error . 

The UI 200 has a fax-only mailbox so that callers can 
deposit directly a fax into their recipient's mailbox 
without first ringing the handset of the recipient. A 
caller can dial the fax-only mailbox number and press a 
start button on his or her fax machine. This feature is 
referred to as walk-away- fax. 

The resource manager 270 assigns various conference 
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resources in a multi-node system, where a multi-node system 
can consist of multiple UI's within a single UI, or can 
consist of a number of UI's connected in tandem. The 
resource manager 270 provides inter-node time slot 
5 management for conference calls. 

As with the pager manager 280, the facsimile manager 
280 and the resource manager 270 communicate data from the 
external facsimile machine 285 and telephone 275, 
respectively, to the interface control module 300. 

10 Information from all three managers, 270, 280 and 290, 

becomes available to the interface control module 300 for 
later access by the user. Although Fig. 6 illustrates an 
embodiment in which the user communicates via a touch-tone 
pad, resource managers, facsimile managers and pager 

15 managers can also be incorporated into a system in which a 
user communicates via speech. 

To support the management of facsimiles, the universal 
interface 200 must contain facsimile drivers. The UI 200 
provides fax capabilities by integrating facsimiles through 

20 the use of the CP12 board available from Dialogic. The 
board and drivers may be interfaced through a fax 
application manager. 

Fig. 6 illustrates the universal interface with 
personal information manager PIM synchronization 

25 capabilities. The PIM 265 manages a user's address book, 
calendar, and to-do list. 

The address book contains entries with fields such as 
name, title, company address, business phone, home address, 
home phone, mobile phone, facsimile number, e-mail address, 

30 web page address and notes. 

The calendar keeps track of appointments. It has 
features such as subject /agenda, location, date/time, and 
appointment length. 

The to-do list is used to keep track of tasks that need 

35 to be accomplished. Each to-do list entry has fields such 
as subject /agenda, priority, due date/time, and reminder 
notes. The UI can page, call, or send e-mail with to-do 
list contents. 



21 



wo 01/76212 



PCT/USOO/01547 



In the embodiment of Fig. 6, the user communicates with 
the universal interface 200, and therefore with the 
information systems, via a touch- tone pad. This embodiment 
differs from the embodiment of Fig. 4, in that the 
5 information system is a computer 262, to which a PIM 265 can 
be connected. 

In most PIM's, such as the *'PalmPilot" or **Palm III," 
available from 3Com Corporation of Santa Clara, California, 
a synchronization process takes place from time to time with 

10 the personal computer 600. Using the "PalmPilot" as an 
example, this synchronization process is accomplished by 
placing the "PalmPilot" in a cradle, which is connected to 
the computer 252 through a serial cable; and pressing a 
*'HotSync" button on the cradle. The synchronization process 

15 sends information from the "PalmPilot" device to the 

computer 262, thereby synchronizing databases on both units. 

In one embodiment, the UI 200 synchronizes the PIM 265 
with the universal interface -200 at the same time the user 
is synchronizing the PIM 265 with computer 262. In this 

20" embodiment, the UI 200 senses or intercepts the 

synchronization signal. When the UI 200 detects that the 
PIM 265 is requesting synchronization, the UI 200 
communicates with the computer 262 and imports the 
synchronization infoannation therefrom. In this way, a 

25 remote server connection is made any time there is a 

synchronization event, thereby making the most current 
synchronized database available via a wireless phone. 

Alternatively, the universal interface 200 can 
periodically synchronize its database with the computer 262. 

30 In this embodiment, the universal interface 200 periodically 
communicates with computer 262 and requests information on 
the last PIM 265 synchronization. If the computer 262 
indicates that it has synchronized with the PIM 265 since 
the last communication with the universal interface 200, the 

35 universal interface 200 sends control logic to the computer 
262 to retrieve the most recent data. 

The personal information synchronization can take place 
over the web. Further, the data sent via the Internet can 
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be' encrypted. Additionally, the data may be sent in 
compressed form or in incremental form when only new or 
updated data is transmitted. 

Fig. 7 illustrates in block diagram form, the structure 
5 of the ICM 300. In the embodiment of Fig. 1, control code 
310 receives as input commands from the speech-to-command 
converter 220 of Fig. 3 or the tone -to- command converter 250 
of Fig. 4. The control code 310 detects the type of 
information system 110 to be accessed, and then accesses a 

10 model 1 through N 320, 330, which corresponds with the 
particular information system 110. Each of the model 1 
through N 320, 330 provides a mapping of the input commands 
to commands recognized by the information system 110. The 
control code 310 looks up the chosen model, 320 or 330, the 

15 information system command that responds to the user input. 
The control code 310 then sends the command to the 
information system 110 and retrieves information therefrom. 

Figs. 8 and 9 illustrate in more detail the output 
switch 400 and the speech-to-text routing switch 500, 

20 respectively. The output switch 400 shown in Fig. 8 has 
three inputs. The first input receives, speech from the 
speech-to-text routing switch 500 (Pigs. 4-5) . This input 
represents speech data retrieved from the information system 
110. The second input- receives the speech output of the 

25 text-to-speech converter 230. This input is used by the 
output switch 400 if the data from the information system 
110 is text which needs to be converted to speech by the 
text -to- speech converter 230. The third input to the output 
switch 400 is a switch control. If the interface control 

30 module 300 detects that the data coming from the information 
system 110 is text, it sends a switch control signal to 
switch 440, toggling switch 440 and allowing the text from 
the information system 110, after being converted by the 
text-to-speech converter 230, to flow to the output of the 

35 universal interface 200. Otherwise, if the interface 
control module 300 detects that the data from the 
information system 110 is speech, then the interface control 
module 3 00 sends a switch control signal to the switch 440 
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which sets the switch 440 in an untoggled state, thus 
routing the speech from the information system 110 through 
to the output of the universal interface 200. 

When the speech first enters the output switch 400, it 
5 is received by a detector 410. The detector 410 detects 
whether the speech is analog or digital in form. If the 
speech is in an analog form, the detector 410 sends the 
speech to switch 430. It also asserts the switch tontrol 
input to switch 430, throwing the switch to a position that 

10 allows the speech input into switch 430 to be sent to the 
switch 440. If, on the other hand, the speech received by 
the detector 410 is digital, the digital speech is sent to 
sound card 420. At the same time, the detector 410 sends a 
switch control input to the switch 430, routing the output 

15 from sound card 420 through to switch 440. 

Fig. 9 illustrates the speech-to-text routing switch 
500. This speech-to-text routing switch 500 receives data 
from the information system 110. This data can either be in 
text or speech format. If the data from the information 

20 system 110 is in speech format, the speech- to- text routing 
switch 500 routes the speech directly to the output switch 
400. If, on the other hand, the data from the information 
system is in text format, the speech-to-text routing switch 
500 is opened. In this instance, the text data is not 

25 routed to the output switch 400 through the speech-to-text 
routing switch 500. 

The interface control module 300 controls the state of 
the switch 520 in the speech- to- text routing switch 500. In 
the event that the interface control module 300 detects that 

30 the data from the information system 110 is speech, it sends 
a control signal to the speech-to-text routing switch 500, 
throwing the switch 520 to a closed state. When the 
interface control module 300 detects text data from the 
information system 110, it sends the control signal to the 

35 speech-to-text routing switch 500, to maintain the switch 
520 in an open position. When the data from information 
system 110 is text, the text from the information system 110 
is received by the interface control module 300 and routed 
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through the text-to-speech converter 230 to the output 
switch 400. This text, which has been converted to speech 

by the text -to- speech converter 230, is then" sent to the 

output of the universal interface 200. 
5 Turning now to Fig. 10, different methods in which a 

user can access various information systems 110 and 650, 
through various universal interfaces 200 and 62 0 are 
illustrated. In one embodiment, the user can access the 
information system 110 directly through a universal 

10 interface 200 through the use of a wireless telephone 105. 
In another embodiment,' the universal interface 620 resides 
in a telephony server 610. In this embodiment, the 
universal interface 620 can access an information system 
650. The universal interface 620 retrieves information from 

15 the information system 650 periodically. When the user 
communicates with the telephony server 610 through a 
wireless telephone 670, the user can retrieve previously 
retrieved information from the information system 650. This 
previously retrieved information is scored in memory 660. 

20 Alternatively, and in a third embodiment, the user can 
interface with. the telephony server 610 to retrieve 
information in real-time. In this embodiment, the telephony 
server 610 acts as a conduit to allow connection between the 
wireliess telephone 670 and the universal interface 200. 

25 From the user's standpoint, the operation is the same as if 
the user had accessed universal interface 200 directly. 

As previously discussed, the universal interface 200 
can operate in a nonreal-time mode. This mode is 
illustrated in Figs. 11 and 12. The interface control 

30 module 3 00 communicates with one or more information systems 
110 and 710. The interface control module 300 periodically 
retrieves data from these information systems. The 
interface control module 300 then stores the retrieved data 
in an internal or an external memory 660. If data retrieved 

35 from the information system 110 or 710 is in speech format, 
the interface control module 300 converts the speech to text 
before storing the data in memory 660. Once the data is 
stored in memory 660, the user can access previously stored 
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data at any time. In the particular embodiment illustrated 
in Fig. 12, the user may access the universal interface 
using a touch- tone pad. In this case, the touch- tone input 
is converted to text via a tone -to -command converter 250 and 
5 is then input into the interface control module 300. The 
interface control module 3 00 interprets the commands from 
the user and retrieves the appropriate data from memory 660. 
Because the data in memory 660 is in text format, the 
interface control module 300 sends the data to a text-to- 

10 speech converter 230, This text-to-speech converter 230 
converts the data into speech and provides the data in 
speech format to the user. 

The embodiment of Fig. 11 operates similarly to that in 
Fig. 12 except that the UI 200 is designed to receive speech 

15 as input . 

In one embodiment of the invention, the universal 
interface contains a language converter 720 (Fig. 12) , which 
has the additional capability of translating electronic mail 
messages, and other data, which is stored in a different 

20 language, such as Spanish, French, among others. The 

language converter 720 first sends the first few lines of an 
e-mail or message to a module called the "language 
identifier." The language identifier analyzes the words 
used by looking them up in a list of often used and typical 

25 words for all languages under consideration. For example, 
words often used in the English language might include 
■''the,'' ''and," and "in." The converter tpillies the results, 
and determines the appropriate language. 

If the language used is different from that spoken by 

30 the user (whose language can be ascertained by the language 
identifier), then the language converter 720, before 
communicating the language to the user, converts the data to 
the user's language. 

The universal interface may also monitor the language 

35 in which the data is stored. If the language converter 720 
detects a change in the language, the universal interface 
200 back-tracks a specified number of words and attempts to 
provide information in the language spoken by the user. 
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This process of changing languages mid- stream can occur 
multiple times within one transmission. 

In another embodiment, the universal interface accesses 
a spell converter 720 which spells out words that the user 
5 indicates are unintelligible. This embodiment compensates 
for pronunciation errors by the text-to-speech converters. 
These errors may occur because either the correct 
pronunciation falls into some grammatical exception or 
because the text being converted contains typographical 
10 errors . 

In this embodiment, the universal interface receives a 
command such as a pound (#) touch- tone from the user 
telephone indicating that the user has. received 
unintelligible words. The universal interface 200 halts the 

15 text -to- speech converter 230 and restarts communication at a 
retry point a specified number of words back from stopping 
- point where the communication had ceased. When the 
universal interface 200 resumes communication, it accesses 
the spell converter 730 which tracks the text that was sent 

20 to the text -to- speech converter. The spell converter 730 
provides, to the Universal Interface, the spelling for a 
predetermined number of words. The universal interface 200 
then starts out by spelling a predetermined number of words 
from the restart point. The universal interface 200 then 

25 resumes communication at or near the same point where the 

user had indicated that the speech was unintelligible. Both 
the language converter 72 0 and the spell converter 73 0 may 
be modules internal to the interface control module 3 00. 
As discussed above, depending upon the user's needs, 

30 the universal interface 200 can retrieve information from 

one or more information systems. These systems may include, 
for example, electronic mail, voice mail, websites, or 
personal information managers. 

Regardless of the information system type, the 

35 universal interface 200 operates with one or more servers to 
. provide the appropriate access. An e-mail server sends and 
receives messages using Internet Message Access Protocol 
(IMAP4) and Post Office Protocol/Simple Mail Transfer 
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Protocol (P0P3/SMTP) protocols, where SMTP is the Internet 
standard for transmitting and forwarding messages, POP is 
the Internet standard for retrieving e-mail, and IMAP is the 
''next generation" Internet standard for retrieving e-mail. 
The e-mail server also services Internet voice and fax 
messages. 

In one embodiment, the e-mail server uses the same 
message store that is used for voice messages. Further, 
shared messages are not replicated. 

IMAP stands for Internet Message Access Protocol. It 
is a method of accessing e-mail or bulletin board messages 
that are kept on a mail server. It permits a ''client" e- 
mail program to access remote message stores as if they were 
local. E-mail stored on an IMAP4 server can be manipulated 
from a desktop computer at home, a workstation at the 
office, and a notebook computer while traveling, without the 
need to transfer messages or files back and forth between 
these computers. IMAP's ability to access messages (both 
new and saved) from more than one computer is important as 
reliance on electronic Messaging and use of multiple 
computers is a common mode of operation for the mobile user. 
IMAP4 supports an offline access to the message store 
followed by a resynchronization of the message store with 
the server. 

The IMAP4 protocol includes operations for creating, 
deleting, and renaming mailboxes; checking for new messages; 
permanently removing messages; setting and clearing flags; 
server-based RFC-822 and MIME parsing, and searching; and 
selective fetching of message attributes, texts, and 
portions thereof for efficiency. Note that MIME is an 
Internet standard for Messaging formatting. MIME allows for 
the attachment of various files to e-mail messages. These 
attachments might include text, voicemail, or faxes. 

The e-mail directory server utilizes a X.500 directory 
that stores user addresses, provides routing information, 
and unifies e-mail components in the environment. The e- 
mail directory server uses the following protocols to 
provide a distributed directory service: LDAP (Lightweight 
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Directory Access Protocol) for client access- -allows for the 
publishing and access of directories such as phone 
directories, DAP (Directory Access Protocol) for message 
routing information, DSP (Directory System Protocol) for 
forwarding requests to remote directory servers and 
obtaining results, DISP (Directory Information Shadowing 
Protocol) for replicating information between servers, and 
HTTP (one of two standard foundations for the World Wide 
Web) for access from Web browsers. The Web/Directory 
gateway facilitates access to LDAP and X.500 Directory 
information from a standard web browser, and utilities to 
import and export data to other applications, including user 
registration from other mail systems. 

Mailbox management must be reconciled between the voice 
and e-mail mailbox utilities. The E-mail X.500 mailbox 
structure and name service must be preserved, as they are 
integral to e-mail standards and protocols. In one 
embodiment, the voice mailbox tools are used as the primary 
mailbox management utilities. Voice mailbox creation would 
be exported to the X.500 e-mail box schema. Although there 
will be replication of the mailbox data between the voice 
and e-mail boxed structures, the message store is common 
between the voice and e-mail servers. Further, event 
notifications may be provided for messages received while 
the user is in a session. 

Another server may service the world wide web. Using 
Active X and Java applets and script technology, one can 
access information over the Internet using a standard web 
browser. The information accessible over the web includes 
information associated with Internet web sites such as 
stock, weather, business information, as well as 
synchronized information from the user's PIM, such as 
calendar, address book and to-do list. 

The web server uses COM interfaces to retrieve PIM 
information, e-mail, voice mail and fax messages. These 
operations may be performed locally on the web server. 

In an embodiment which handles voice mail systems, the 
UI 200 provides an interface to various legacy voice mail 
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systems such as systems from Octel or Centigram. The legacy 
system may forward voice and fax messages to the UI 200 
using VPIM (Voice Profile for Internet Messaging) . VPIM is 
based on MIME, an Internet standard for message formatting, 
5 which allows voice and fax mail systems to exchange messages 
over the Internet • 

The UI may have fewer voice mail features than the 
legacy systems • In such a case, the UI maps strings and 
features that are supported and accesses data according to 

10 its perhaps limited abilities. 

The UI provides the use of a voice or DTMF dial -out to 
the legacy system while preserving the virtual assistant 
functionality. The user can change parameters of the 
universal interface, and can also expand the UI 

15 capabilities, through either the telephone or the Web. 

Configuration changes over the web may be either menu or 
form driven. Changes over the phone may be menu driven . 

It should be noted that although the input to the 
universal interface 200 has been illustrated as either tone 

20 or speech, the invention can accept as input either speech, 
tone, or both speech and tone. Further, while the invention 
has been shown and described with reference to an embodiment 
thereof, those skilled in the art will understand that the 
above and other changes in form and detail may be made 

25 without departing from the spirit and scope of the following 
claims. 
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WHAT IS CLAIMED IS: 

1. A universal interface for accessing one or more 
information systems from a telephone, each of the 

5 information systems being adapted to generate text data or 
speech data in response to a query, the universal interface 
comprising: 

an interface control module for receiving a sound-based 
command from a user and for sending a corresponding machine 
10 readable command to a source selected from one of the 

information systems, the interface control module receiving 
data from the source in response to the corresponding 
machine readable command; 

a text -to- speech converter coupled to the interface 
15 module for converting text data received by the interface 

control module to a converted speech segment and playing the 
converted speech segment for the user; and 

a router coupled to the interface module for receiving 
speech data from the source and playing a speech segment 
20 corresponding to the speech data for the user. 

2. The universal interface of claim 1, wherein the 
telephone is 

wireless . 

3. The universal interface of claim 1, wherein the 
25 sound-based command is provided by the user's voice. 

4. The universal interface of claim 1, wherein the 
sound-based command is specified using one or more telephone 
touch- tones . 

5. A universal interface for accessing one or more 

30 information systems from a user telephone, the universal 
interface comprising: 

an input converter for converting input from the user 
telephone to commands; 

an interface control module coupled to the input 
35 converter for receiving the commands from the input 

converter; determining one of the information systems to be 
accessed; converting the commands to commands recognizable 
by the information system; forwarding the converted commands 
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to the information system; receiving data from the 
information system; detecting the form of the data from the 
information system; and storing information relating to a 
current state of the system; 

a speech-to-text routing switch coupled to the 
interface control module for receiving data from the 
information system and control data from the interface 
control module; 

a speech- to -command converter coupled to the interface 
control module for converting speech to commands, wherein 
the speech- to- command converter coupled to the speech-to- 
command routing switch to receive speech and to forward 
commands to the interface control module; 

an output switch coupled to the interface control 
module and the speech- to- text routing switch for receiving 
speech from the speech-to-text routing switch for receiving 
a control input from the interface control module, and for 
forwarding speech from the speech-to-text routing switch to 
the user telephone; and 

a text -to- speech converter coupled to the output switch 
for receiving text from the interface control module, 
converting the text to speech, and forwarding the speech to 
the output switch to deliver speech to the user telephone. 

6. The universal interface of claim 5, wherein the user 
telephone is a mobile telephone. 

7. The universal interface of claim 5, wherein the input 
converter comprises a speech-to-command converter. 

8. The universal interface of claim 5, wherein the input 
converter comprises a tone -to -command converter. 

9. The universal interface of claim 5, wherein the 
interface control module integrates the data from the 
information system periodically or manually under user 
control, the interface control module further comprising: 

means for retrieving data from the information systems; 
means for determining antecedent comparable relevance 
of the data; 

means for updating all of the data to reflect the most 
recent data; 
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means for linking relevant data; and 

means for exporting the linked data to the information 
system. 

10. The universal interface of claim 9, wherein the 
information systems comprise two or more of the following: 

a calendar; 
a to-do list; 
an address book; 
voice mail; 
e-mail; and 
a web site. 

11. The universal interface of claim 5, wherein the 
interface control module integrates and synchronizes (i) a 
database of a personal information manager, (ii) a database 
residing on a personal digital assistant, and (iii) a 
database residing in the universal interface, the universal 
interface coupled to a computer on which the personal 
information database resides, wherein, a computer further 
comprises : 

a first input terminal for receiving data from the 
personal digital assistant; 

a second input terminal for receiving data from the- 
universal interface ; 

a sensor for detecting a synchronization event 
triggered by a user requesting synchronization of the 
database of the personal digital assistant with the database 
of the personal information manager; 

an electronic mail system coupled to the Internet; and 

control logic coupled to receive the synchronized event 
from the sensor and to transmit data, over the electronic 
mail system, to the interface control module, wherein the 
control logic updates the data in each of the databases to 
reflect the most recent data entered into any database. 

12. The universal interface of claim 11, wherein the 
synchronization information is sent to the universal 
interface in its entirety, in compressed form, or in 
incremental form. 

13. The universal interface of claim 5, wherein the data 
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sent from the information system to the universal interface 
is sent over the Internet. 

14 . The universal interface of claim 13 , wherein the data 
sent is encrypted. 

15. The universal interface of claim 5, wherein the output 
switch further comprises: 

a detector for determining whether the speech from the 
speech- to- text routing switch is digital or analog; 

a sound card coupled to the detector to receive digital 

data; 

a first speech routing switch for receiving a control 
input from the detector, and speech from the detector or 
speech from the sound card; 

a second speech routing switch for receiving the 
control input from the interface control module and speech 
from the first speech routing switch or speech from the 
text-to-speech converter, the second speech routing switch 
forwarding speech to the user telephone . 

16. The universal interface of claim 5, wherein the 
information system comprises at least one of: a dual tone 
multiple frequency (DTMF) driven voice mail system, a voice 
driven voice mail system, an electronic mail system, a web 
site, and a personal information manager. 

17. The universal interface of claim 5, wherein the system 
determines one of the following: 

whether the voice commands are being received from a 
user telephone; 

the information system to be accessed; 

whether the voice commands, after being converted to 
commands, are recognizable to the information system; 

whether the converted commands have been forwarded to 
the information system; 

whether data has been received from the information 
system; 

whether data from the information system is speech or 

text ; 

the state of the speech-to-text routing switch; and 
the state of the output switch. 
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18. The universal interface of claim 5, wherein the 
interface control module further comprises: 

a model for converting commands from the telephone into 
commands recognizable by the information system; and 
5 a translator coupled to the input converter for 

retrieving the model corresponding to the information 
system to be accessed and for converting the commands to 
commands recognizable by the information system. 

19. The universal interface of claim 5, wherein the 
10 universal interface further comprises: 

means coupled to the input converter for signaling that 
the user telephone has received unintelligible words; 

means for restarting communication to the text-to- 
speech converter at a point a specified number of words back 
15 from the point at which the communication ceased; 

means for forwarding the first specified number of 
words by spelling the words out; and 

means for continuing forwarding the data after the 
specific number of words. 
20 20. The universal interface of claim 5, wherein the 
universal interface further comprises: 

means for detecting a first language in which the 
commands from the user telephone are received; 

means for detecting a second language associated with 
25 the data received from the information system; and 

means for converting the data from the information 
system into the first language. 

21. The universal interface of claim 20, wherein the 
universal interface further comprises means for detecting 

30 more than one languages within a single fragment of data. 

22. The universal interface of claim 5, further comprising 
a resource manager for establishing conference bridges to an 
external telephone having a telephone number, wherein: 

the interface control module detects, from the commands 
35 from the input converter, whether a conference bridge 
request has been made; 

the interface control module retrieves the telephone 
number of the external telephone to establish the conference 
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bridge; 

the interface' control module forwards the telephone 
number to the resource manager; and 

the resource manager establishes a telephone connection 
5 with the external telephone. 

23. The universal interface of claim 5, further comprising 
a facsimile manager for sending facsimiles to one or more 
facsimile machines, wherein: 

the interface control module detects, from the commands 
10 from the input converter, whether a facsimile request has 
been made; 

the interface control module retrieves a telephone 
number of a designated facsimile machine; 

the interface control module foirwards the data and the 
15 telephone number to the facsimile manager; and 

the facsimile manager faxes the data to the designated 
facsimile machine. 
■ 24. The universal interface of claim 5, wherein the 
universal interface further comprises a pager manager for 
20 sending pager messages to one or more pagers, wherein: 

the interface control module detects, from the commands 
from the input converter, whether a pager request has been 
made ; 

the interface control module retrieves data to be 
25 forwarded to the pager; 

the interface control module retrieves a telephone 
number of a designated pager to which the data is to be 
forwarded; 

the interface control module forwards the data and the 
30 telephone number to the pager manager; and 

the pager manager forwards the data to the designated 
pager . 

25. A universal interface for accessing data from one or 
more information systems comprising: 
35 an input converter for receiving inputs from the user 

telephone and converting the inputs into commands; 

an interface control module coupled to the input 
converter comprising: 
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means for periodically requesting data from the 

information systems; 

means for retrieving data from the information 

systems- 
means for detecting the form of data from the 

information system; 

means for converting the data that is speech to 

commands ; 

means for storing the data for later access by a 
user telephone; and 

means for storing information representing a 
current state of the system; and further comprising: 

means for receiving commands from the input 
converter and means for detecting which of said stored 
information is sought to be accessed; 
a text-to-speech converter coupled to the interface 
control module for receiving said requested stored text from 
the interface control module, converting the text to speech, 
and forwarding the speech to the user telephone. 

26. The universal interface of claim 25, wherein the user 
telephone is a mobile telephone. 

27. The universal interface of claim 25, wherein the input 
converter comprises a speech- to- command converter. 

28. The universal interface of claim 25, wherein the input 
converter comprises a tone -to -command converter. 

29. The universal interface of claim 25, wherein the 
interface control module integrates the data from the 
information system periodically or manually under user 
control, the interface control module further comprising: 

means for retrieving data from the information systems; 
means for determining antecedent comparable relevance 
of the data; 

means for updating all of the data to reflect the most 
recent data; 

means for linking relevant data; and 

means for exporting the linked data to the information 
system. 

30. The universal interface of claim 29, wherein the 
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information systems comprise two or more of the following: 
a calendar; 
a to-do list; 
an address book; 
voice mail; 
e-mail; and 
a web site. 

31. The universal interface of claim 25, wherein the 
interface control module integrates and synchronizes (i) a 
database of a personal information manager, (ii) a database 
of a personal digital assistant, and (iii) a database 
residing in the universal interface, the universal interface 
coupled to a computer on which the personal information 
datsU^ase resides, said computer comprising: 

a first input terminal for receiving data from the 
personal digital assistant; 

a second input terminal for receiving data from the 
universal interface; 

a sensor for detecting a synchronization event, wherein 
the event is triggered by a user requesting synchronization 
of the database of the personal digital assistant with the 
database of a personal information manager; 

an electronic mail system coupled to the Internet; 

control logic coupled to receive detection information 
from the sensor and to transmit data, via the electronic 
mail system, to the interface control module, wherein the 
control logic updates the data in each of the databases to 
reflect the most recent data entered into any one database. 

32. The universal interface of claim 31, wherein the 
synchronization information is sent to the universal 
interface in its entirety, in compressed form, or in 
incremental form. 

33. The universal interface of claim 25, wherein the data 
sent from the information system to the universal interface 
is sent over the Internet. 

34. The universal interface of claim 33, wherein the data 
sent over the Internet is encrypted. 

35. The universal interface of claim 25, wherein the 
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information system comprises at least one of: a dual tone 
multiple frequency (DTMF) driven voice mail system, a voice 
driven voice mail system, an electronic ' mail system, a web 
site, and a personal information manager. 
5 36. The universal interface of claim 25, wherein the system 
determines one of the following: 

whether the voice commands are being received from a 
user telephone; 

the information system to be accessed; 
10 whether the voice commands, after being converted to ^'^ 

commands, are recognizable to the information system; 

whether the converted commands have been forwarded to . 
the information system; 

whether data has been received from the information 
15 system; 

whether data from the information system is speech or 

text ; 

• . the state of the speech-to-text routing switch; and 

the state of the output switch. 
20 37. The universal interface of claim 25, wherein the 
interface control module further comprises: 

one or more models containing commands recognizable by 
the information system; 

control logic for accessing the model that corresponds 
25 with the information system to be accessed, converting 

control commands to commands recognizable by the information 
system, and forwarding the converted commands to the 
information system. 

38. The universal interface of claim 25, wherein the 
30 universal interface further comprises: 

means coupled to the input converter for signaling that 
the user telephone has received unintelligible words; 

means for restarting communication to the text-to- 
speech converter at a point a specified number of words back 
35 from the point at which the communication ceased; 

means for forwarding the first specified number of 
words by spelling the words out; and 

means for continuing forwarding the data after the 
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specific number of words. 

39. The universal interface of claim 25, wherein the 
universal interface further comprises: 

means for detecting a first language in which the 
commands from the user telephone are received; 

means for detecting a second language associated with 
the data received from the information system; and 

means for converting the data from the inf oinnation 
system into the first language. 

40. The universal interface of claim 39, wherein the 
universal interface further comprises means for detecting 
more than one language within a single fragment of data. 

41. A method for providing data from one or more 
information systems to a user telephone, comprising the 
steps of : 

converting speech from the user telephone to commands; 

determining the information system to be accessed; 

converting the text to commands recognizable by the 
information system; 

forwarding the converted commands to the information 
system; 

receiving data from the information system; 
detecting the form of the data from the information 
system; 

converting non- speech data from the information system 
into speech; 

forwarding the speech data to the user telephone; and 
storing information relating to the current state of 
the system. 

42. The method of claim 41, wherein converting speech from 
the user telephone further comprises converting speech from 
a mobile telephone. 

43. The method of claim 41, wherein converting speech from 
the user telephone to commands further comprises converting 
speech to commands. 

44. The method of claim 41, wherein converting speech from 
the user telephone to commands further comprises converting 
dual -tone -multiple -frequency (DTMF) information to commands. 



40 



wo 01/76212 



PCT/USOO/01547 



45. The method of claim 41, further comprising 
synchronizing the data from the information systems by 
periodically, or upon command: 

retrieving data from the information systems; 
5 comparing the data to determine which data is most 

recent ; 

updating all of the data to reflect the most recent 

data ; 

linking relevant data; and 
10 exporting the linked data to the information systems. 

46. The method of claim 45, wherein the information systems 
comprise two or more of the following: 

a calendar; 
a to-do list; 
15 an address book; 

voice mail; 
e-mail; and 
a web site. 

47. The method of claim 41, further comprising integrating 
20 and synchronizing (i) a database of a personal information 

manager, (ii) a database of a personal digital assistant, 
and (iii) ci database residing in a universal interface, by: 

receiving data from the personal digital assistant; 

receiving data from the universal interface; 
25 detecting a synchronization event, wherein the event is 

triggered by a request for synchronization of the database 
of the personal digital assistant with the database of a 
periBonal information manager; 

transmitting data, via an electronic mail system, to 
30 the interface control module; and 

updating the data in each of the databases to reflect 
the most recent data entered into any one database. 
48 i The method of claim 47, wherein the synchronization 
information is sent to the universal interface either in its 
35 entirety, in compressed form, or in incremental form. 

49. The method of claim 41, further comprising sending the 
data from the information system to the universal interface 
over the Internet. 
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50. The method of claim 49,. further comprising encrypting 
the data from the information system before sending the 
encrypted data over the Internet . 

51. The method of claim 41, wherein the information system 
is comprised of at least one of: a voice mail system, and 
an electronic mail system, a web site, and a personal 
information manager. 

52. The method of claim 41, wherein storing information 
relating to the current state of the system further 
comprises the steps of determining: 

whether the voice commands are being received from a 
user telephone ; 

the information system to be accessed; 

whether the voice commands, after being converted to 
commands, have been converted into commands recognizable by 
the information system; 

whether the converted commands have been forwarded to 
the information system; 

whether data has been received from the information 
system; 

whether data from the information system is speech or 

text ; 

the state of the speech- to- text routing switch; and 
the state of the output switch. 

53. The method of claim 41, wherein converting the text to 
commands recognizable by the information system further 
comprises accessing one or more models for converting 
commands from the telephone into commands recognizable by 
the information system. 

54. The method of claim 41, further comprising: 
receiving a command from the user telephone signaling 

that the user telephone has received unintelligible words; 

ceasing communication to the user telephone; 

restarting communication to the user telephone at a 
point a specified number of words back from the point at 
which the communication ceased; 

forwarding the first specified number of words by 
spelling the words out; and 
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continuing to forward the remainder of the data. 
55. The method of claim 41, further comprising: 

detecting the language in which the commands from the 
user telephone are received; 
5 detecting the language of the data received from the 

information system; 

converting the data from the information system into 
the language in which commands from the user telephone are 
received. 

10 56. The method of claim 55, wherein detecting the language 
of the data received from the information system further 
comprises detecting more than one language within a single 
piece of data. 

57. The method of claim 41, further comprising establishing 
15 conference bridges to one or more external telephones, by: 

detecting, from the commands from the input converter, 
whether a conference bridge request has been made; 

retrieving a telephone number of the external telephone 
to which the conference bridge is to be established; 
20 forwarding the telephone number to a resource manager; 

and 

establishing, through the resource manager, a telephone 
connection with the external telephone, thereby opening a 
line of communication between the user telephone and the 
25 external telephone. 

58. The method of claim 41, further comprising sending 
facsimiles to one or more facsimile machines, by: 

detecting, from the commands from the input converter, 
whether a facsimile request has been made; 
30 retrieving data to be faxed; 

retrieving a telephone number of a designated facsimile 
machine to which the data is to be forwarded; 

forwarding the data and the telephone number to a 
facsimile manager; and 
35 with the aid of the facsimile manager, faxing the data 

to the designated facsimile machine. 

59. The method of claim 41, further comprising sending 
pager messages to one or more pagers, by: 
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detecting, from the commands from the input converter, 

whether a pager request has been made; 

retrieving data to be forwarded to a pagers- 
retrieving a telephone number of a* designated pager to 

which the data is to be forwarded; 

forwarding the data and the telephone number to a pager 

manager ; and 

with the aid of the pager manager, forwarding the data 
to the designated pager. 

60. A method for accessing data from one or more 
information systems comprising the steps of: 

retrieving data from the information system; 

detecting the form of data from the information system; 

converting the data that is speech to text; 

storing the data for later retrieval; and 

storing information representing the current state of 
the system. 
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US 5.524.137 A (RHEE) 04 June 1996. abstract and Fig. 4. 

US 5.297,040 A (HU) 22 March 1994. col. 13. line 55-col. 14, line 
2. 

US 5.689,616 A (LI) 18 November 1997. abstract. 

US 4,882.681 A (BROTZ) 21 November 1989. 

US 5,268,839 A (KAJI) 07 December 1993. 

US 5,633.916 A (GOLDHAGEN et al.) 27 May 1997. 

US 5,647,002 A (BRUNSON) 08 July 1997. abstract and Fig. I. 

US 5,666,400 A (MCALLISTER et al.) 09 September 1997, 

US 5,675.507 A (BOBO. II) 07 October 1997. 

US 5.708,804 A (GOODWIN et al.) 13 January 1998. 

US 5,790,974 A (TOGNAZZINl) 04 August 1998 

US 5.737,395 A (IRRIBARREN) 07 April 1998 

US 5,822,404 A (CAVE) 13 October 1998. 

US .5.859.898 A (CHECCO) 12 January 1999 

US 5.875,422 A (ESLAMBOLCHl et al.) 23 February 1999. 
abstract and Fig. 1 . 

US 5.884,262 A (WISE el al.) 16 March 1999. 
US 5,915,001 A (UPPALURU) 22 June 1999, abstract. 
US 5,953.393 A (CULBRETH et al.) 14 September 1999. 
US 5,960.1 13 A (EVEN et al.) 28 September 1999, abstract. 

US 5,960,406 A (RASANSKY et al.) 28 September 19W. 

US 5,901,366 A (HENRICK et al.) 3.1 Novumbor I WD 
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US 5.999.61 1 A (TATCHELL et al.) 07 December 1999. abstract. 
US 6,016,478 A (ZHANG et al.) 18 January 2000, abstract. 

US 6.034,62 1 A (KAUFMAN) 07 March 2000, abstract. 



COLE, B. Lotus hooks Organizer to the Web, Network World, 
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PCT/USOO/01547 



I Ohsf.rvutiiiiis %vlu'ri' certMiii cluiiiis wvrv Touiul uiiscurcliublf (C'lHitiiiuuti if item I nf first slifct) 

Tills imcnialional rqv»rt has ivtl Ihxii cstahtisliixJ in rvsjuxl nl ccrUiin cbiiiis uiulcr Article I7<2)(;0 lor U)c rullnwiuj: rcasmis. 

I. I I Claims Nos.; 

because ihey rclale In siibjccl matter not required Id be searched by this Aiiihnriiy.' iianicly: 



□ 



Claims Nos. : 

because they relate lo (larts of the international application that do not comply with the prescril^eJ requirciiK'iUs such 
an extent that no meaningful international search can be ciirined out, siHscifically: 



3. I I Claims Nos.: 

because Ihey are dc[vndcnt ckiiius and are nol dralWd in acciirdiUKc with the seotnd and lliird suiiciwus oi' Kuk> 0.4ial. 



II Observations where unity of invention is lacking (<'uiitinuatiiin ul' item 2 of first sheet) 



This Inteniational Sectrchin^ Authority found multiple inveniions in this internalional application, as folltiws. 
PleiJse See Extra Sheet. 



1 . j x] As ail required additional search fees were timely paid by ihc applicaiU. this international search report covers alt searchable 

claims. 

2. I I As all searchable claims could be searcited without effort justifying an addilional fee. Ibis Authority did iml Inviic payment 

of any additional fee. 

3. [ [ As iinly some of Ihe required additional search fees were timely paid by the applicant . tins inleritational search ivpon covers 

only thitse claims lor which fees were paid. s|>ecifically claims Nos.. 



4. I I No required additional search fees were timely paid by the applicant. C«>nscqiicitily. this inteniatimial search rcpurt is 
reslricled lo the invenlii)n first meniiimed in the claims: it is covered bv claims No.s.. 



Kemark on I'mtesl [ | The additional .search fees were acciunpanicd by ilic applicant *s prolesl 

I I No pruiesi acctim|i;uiied the payment of additional search fees 
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B. FIELDS SEARCHED 

Miniimnn docuincn!;U"nm searched 
Classificalion Sysicm: U.S. 

379/67.1. 88.01. 88.04. 88.05. 88.06. 88.13. 88.14. 88.15, 88.10. 88.17, 88.22. 88.23. 88.24, 88.25. 93.12: 704/1. 
2. 3, 4. 5. 6, 7, 8. 9, 10. 200. 231. 249. 251. 258. 275, 277; 370/214. 351. 352. 3!53. 354. 355, 356: 455/130. 340. 
557; 709/200. 201. 202. 204. 206. 217. 218, 219 

BOX n. OBSERVATIONS WHERE UNITY OF INVENTION WAS LACKING 
This ISA found mulliple invcntion.s as follows: 

This appUcaiion coniaios the following inventions or group.s of inventions which are not sn linked as in luriu :i sinj:le 
inventive concept under PCT Rule 13.1. In order for all inventions to be seiirched. the appmpriale aildiiimial .scarvli 
ices inusl he paid. 

Group 1. ciaiin(s) 1-24. drawn tn a universal information access 
system using a router. 

Group II. clainus) 25-40. drawn to a univenuil in formal ion access 
system using i^eritidic access. 

Group 111. claim(s) 41-60, dmwn to a universal infonnalii>n 

access system where the information system to he accessed is determined after a speech c«Minnaiul cimunand from Ihc 
user. 



The inventions listed as Groups I-III do not relate to a single inventive concept under PCI* Rule 13. 1 twausc. under 
PCT Rule 13.2. they lack the same or eorrx:.s(XMiding special teehnic;il fe;iuires for the fotluwin^ rca.siins lUuiy 
invention has to he considered in the tlrsl place only in relation to the indet'>endeni claims m an inlernaliiinal apphcaiiun 
and niu ihc dependent claims. The inde^iendcni clainus do nol share certain s|H;cial technical fe:iiurcs. l-or example, ihc 
indepcni claiii) 41 of Group 111 recites the unique special lechnical fe:Jture of deiermining the infonualion sy.^ilem l4» he 
accessed after the user s spoken command. However. Ihc indej>endem claim 25 of Group II recites the unique special 
lechnical feature for alli»wing the user t*i periodically access the tnfonnaliim syslcm. Finally. Ihe independcni claim I 
of Group I recites the unique s}>ecial lechnical fe;ilure (»f a router coupled id the inierfacc module. 
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